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(54) Apparatus and method for specifying presentation of multimedia message components 



(57) A new type of message component (204, 205) 
enables message creators to specify, inside of a mes- 
sage (200), how the body components (202-203) of the 
message are to be presented to message recipients: 
multiple such components enable multiple presenta- 
tions to be specified for a single message. Included in 
the message by the message creator as one or more 
message components (204. 205) of type "presentation", 
each presentation component specifies the order, dura- 



tions, and any concurrency of presentation of the mes- 
sage's body components. Each presentation compo- 
nent is a script that, upon its invocation (300) by the mes- 
sage recipient, executes (302-318) on the recipient's 
message-presentation equipment (102. 107) and 
presents the message's body components to the mes- 
sage recipient in the order and for the durations speci- 
fied, and presents concurrently any body components 
- typically ones expressed in different media -- that are 
specified to be presented concurrently. 
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Description 
Technical Field 

This invention relates to electronic messaging systems, and specifically to multimedia messaging systems. 
Background of the Invention 

Multi-media messaging systems permit a message to comprise one or more message components each expressed 
in any one of a plurality of different media. Commonly-used media for multimedia messaging include audio, fax. text, 
image (graphics), and video. 

Quite often, the different message components have a specific relationship with one another and this relationship 
needs to be preserved dunng message playback. Consider the example of a message that comprises an annotated 
slide presentation. There is a set of images which represent the slides, and a set of voice annotations each corre- 
sponding to each different slide. When the message is played out to the message recipient, it is desirable - if not 
imperative -- that the slides be presented to the message recipient in the correct order and at a desirable rate, and 
that the presentation of each slide be accompanied by a simultaneous presentation of its corresponding voice anno- 
tation. There is therefore a need for an arrangement that effects the specifying and enforcing the order and relationship 
of the slides and voice annotations. 

Moreover, the message sender may wish to provide some recipients of the message with a full presentation, but 
may want to provide other recipients with only an abridged presentation, without having to create two entirely different 
messages. Alternatively, the sender may wish to provide to the message recipient or recipients a choice between the 
full presentation and the abridged presentation without having to provide them with two different messages. 

Unfortunately, known multi-media message authoring systems provide message creators with limited capabilities 
for specifying relationships between message components and enforcing these relationships dunng message playback. 
Illustrative examples of such systems are disclosed in U, S. Patents No. 5,420,801 and 5.388.264, and in Japanese 
Kokai Patent Application No. SHO 60[1 9S5j-30244. Generally, the known systems allow the message sender to specify 
only one presentation arrangement (e.g. . component ordering and timing) for a message. They generally also enforce 
this single presentation arrangement during message playback, and do not allow the recipient to choose between 
different presentation arrangements 

Summary of the Invention 

This invention is directed to solving and meeting these and other disadvantages and needs of the prior art. Generally 
according to the invention, there is provided a new type of message component that enables message creators to 
specify, inside of a message, how (e.g., when, how fast, and in what order) the other components of a message are 
to be presented to message recipients, and that further enables multiple such presentations to be specified for a single 
message. Specifically according to the invention, a messaging method and apparatus receive and store, respectively, 
a message comprising a header component that describes the message, a plurality of body components each carrying 
a portion of message information earned by the message, and at least one presentation component that is separate 
from the header component and the body components and that defines an order and any concurrency of presentation 
of the body components to a message recipient. In response to invocation (e.g., a request to execute) of an individual 
presentation component, the method and apparatus present the body components to the message recipient in the 
order and with any said concurrency defined by the individual presentation component. Preferably the message is a 
multimedia message that comprises a plurality of body components at least some of which are expressed in different 
media, and the individual presentation component defines an order of presentation of the body components, including 
a concurrency of presentation of the body components expressed in different media. In response to the invocation of 
the individual presentation component, the method and apparatus then present the body components in the order 
defined by the individual presentation component, including concurrently presenting the body components expressed 
in the different media. Further preferably, the individual presentation component further defines durations of presen- 
tation of the body components, and the method and apparatus also present the body components for the durations 
defined by the presentation component. 

The invention advantageously enables a message creator to specify the order, concurrency, and preferably also 
duration of presentation of messages. The invention is particularly effective for specifying the concurrency of presen- 
tation of message components of multimedia messages that are expressed in different media. The invention advan- 
tageously allows the presentation, or even multiple presentations, to be specified in the message itsell, without requiring 
any change in conventional message-creation, transmission, or reception systems, by including the presentation spec- 
ification or specifications in the message itself as just another message component or components. The message 
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recipient's message-presenlatton equipment then merely interprets or executes the presentation component in order 
to present the message to the message recipient in the manner desired by the message creator 

These and other advantages and features of the invention will become apparent from the following description of 
an illustrative embodiment of the invention taf^en together with the drawing. 

5 

Brief Description of the Drawing 

FIG. 1 is a block diagram of a messaging network that includes an illustrative implementation of the invention; 
FIG. 2 is a block diagram of a message of the messaging network of FIG. 1 that embodies an illustrative imple- 
^0 mentation of the invention: 

FIG. 3 is a functional flow diagram of execution of a presentation script in response to an "execute presentation" 
command in the network of FIG. 1 : 

FIG. 4 is a functional flow diagram of execution of a "skip** command in the network of FIG. 1 : 
FIG. 5 is a functional flow diagram of execution of a "play" command in the network of FIG. 1: and 
'5 FIG. 6 is a functional flow diagram of execution of a "wait" command in the network of FIG. 1 . 

Detailed Description 

FIG 1 shows a generalized multimedia messaging network 100 It illustratively comprises a plurality of intercon- 

20 nected multimedia messaging systems 101-103. each of which may be the AT&T Intuity® AUDI X® system, for example. 
(\/1essaging systems 101-103 are connected to multimedia workstations 106-108 of messaging-service subscribers. 
Each workstation 106-108 is a general-purpose computer operating under control of programs stored in its memory 
and executed by its processor. Each messaging system 101-103 defines mailboxes of those subscribers whom it 
serves. As shown in FIG. 1. workstation 106. of a human user who is assumed to be a message onginator in this 

2S illustrative example, is connected to messaging system 101. IVlessaging system 101 is identified as the authoring 
system, and defines a mailbox 1 04 for the user of workstation 1 06. designated as the author's mailbox. Also, workstation 
107. of a human user who is assumed to be a message recipient in this illustrative example, is connected to messaging 
system 102. Messaging system 102 is identified as a playback system, and defines a mailbox 105 for the user of 
workstation 107, designated as the recipient's mailbox. Alternatively, workstations 106 and 107 may be served by the 

^0 same messaging system, in which case the authoring system and the playback system will be the same system. Each 
recipient's workstation 107-108 preferably includes and executes a copy of a message management (MM) 1 09 program, 
such as the AT&T Inluity Message Manager. Each messaging system 101-103 is configured conventionally: it is a 
stored-program-controlled machine that comprises a memory 110 for storing data (including subscriber's mailboxes) 
and control programs, a processor 111 for executing the control programs out of memory 110 to control the operation 

3S of the messaging system and provide messaging services, and an input and output (I/O) interface 112. typically com- 
prising a plurality of telephone line and trunk circuits and/or local area network (LAN) or wide area network (WAN) 
interfaces, by means of which processor 111 communicates with the messaging network 100 that lies outside of its 
messaging system 101-103. 

Communications between subscribers in messaging network 100 are effected in the following manner: A message 
■^0 author creates a message in his or her own mailbox 104 via his or her workstation 106, and then sends the message 
to an address or addresses of one or more message recipients. Network 100 delivers and stores the message in the 
mailbox 105 or mailboxes of the one or more recipients. The recipients then effect playback of the message from their 
mailboxes via their workstations 107-108. As described so far messaging network 100 is conventional 

Conventionally, a multi-media message 200 (see FIG. 2) comprises a plurality of message components 201-203. 
The first message component is a message header 201 which carries information that describes the message, includ- 
ing information that defines a message presentation, followed by one or more message-body components, each ex- 
pressed in one of a plurality of different media, that carry the message's information contents. The medium in which a 
message-body component 202-203 is expressed constitutes the component's type. Each component 202-203 may 
further comprise one or more records 206. A record is, for example, one page of a text or a fax component, or one 
50 image of an image component, or a scene (a plurality of frames) of a video component, etc. 

According to the invention, the conventional message configuration is modified such that message header 201 no 
longer carries the message presentation information. Instead, there is defined a new message component type, des- 
ignated as "presentation", for carrying the message presentation information. In respects other than its type, a "pres- 
entation" message component is treaded by messaging network 100 as any other message-body component. A mes- 
55 sage may include one or more "presentation" components, each defining a different presentation of the message This 
message configuration is shown in FIG. 2. 

Each "presentation" component 204-205 is a script that defines the sequence and duration of presentation of 
message-body components or components* records to the message recipient. Illustratively the language for composing 
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the scripts is very simple, and comprises the three commands listed in Table A. 



Table A 



command 


arguments 


description 


play 


component_ID 
component_ID #_records 


plays the whole indicated component, or the 
indicated number of records of the indicated 
component: all play commands in a sequence that 
is not interrupted by a "wait" command are 
executed simultaneously 


wait 


time 


stop playback after the indicated time has elapsed 


componentJD 
component_ID #_records 


stop playback when the indicated component, or 
the indicated number of records of the indicated 
component, has finished playing 


skip 


time 


skip the indicated amount of lime into the 
subsequently identified component 


#_records 


skip the indicated number of records (e.g. . pages, 
images, etc.) into the subsequently-identified 
component 


skip (alternative implementation) 


+ time 
- time 


skip the indicated amount of time forward (+) or 
backward (-) from the present position in the 
subsequently-identified component 


+ #_records 
- #_records 


skip the indicated number of records forward (+) 
or backward (-) from the present position in the 
subsequently-identified component 



The message originator composes a presentation script and includes it in message 200 as a "presentation" com- 
ponent 204-205 in the same manner and using the same tools (e.g., an editor) as he or she would use to compose 
any other message-body text component. 

Using again for illustrative purposes the voice-annotated slide presentation message, assume that each voice 
annotation is a separate message-body "audio" component, while each slide image is a separate record of a single 
message-body "image" component. Assume further that all slides are not necessarily annotated. A presentation com- 
ponent 204 for presenting this entire message may comprise the following script: 
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script comments 



play image l^record display first slide while 

playing first annotation 

play audio_l 
wait audio 1 



skip {^records display second slide for 

play image l^record 15 seconds and simultaneously 

play audio_2 play second annotation 
wait 15 seconds 



skip 2_records 
play image l^record 
wait 15 seconds 



skip 3_records 
play image l^record 
play audio_3 
wait audio 3 



display third slide 
for 1 5 seconds 



display fourth slide 
while playing third 
annotation 



Furthermore, another presentation component 205 for presenting an abridged version of the voice-annotated slide 
presentation message may comprise the following script: 



script 


comments 


skip 2_records 
play image 1 ^record 
wait 15 seconds 


skip the first two slides 
and display the third slide 
for 15 seconds 


skip 5_records 
play image 1 _record 
play audio 5 
wait audio_5 


skip the next three slides 
and display the sixth slide 
while playing the fifth annotation 


skip 3_records 
play image 1 _record 
play audio_3 
wail audio_3 


skip back two slides 

and display the fourth slide 

while playing the third annotation 


skip 13_fecords 


skip the next 10 slides 
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(continued) 



script 



connments 



5 



play image record_14 
play audio_11 
wait 15_seconds 



and display the fourteenth slide 

slide for 15 seconds and 

simultaneously play the eleventh annotation 



When the message recipient accesses a received message 200 in his or her mailbox 105, he or she can view the 
presentation message component 204 or components 204-205 in the same manner and using the same tools (e.g.. 
workstation display screen) as he or she would use to view any other message-body text segment. If and when the 
message recipient elects to have message 200 presented to him or her the message recipient gives an "execute 
presentation" command either to message management 109 of his or her workstation 107 or - if workstation 107 is 
not equipped with message management 109 -- to his or her playback system 102, followed by an identifier of one of 
the presentation components 204-205 of message 200. at step 300 of FIG. 3. (If message 200 contains only one 
presentation component 204. the "execute presentation" command need not be accompanied by a presentation com- 
ponent identifier.) In response, workstation 107 or processor 111 retrieves the indicated presentation component 204 
from message 200 in recipient's mailbox 105. at step 302. gets the first commando! the presentation script from the 
retrieved presentation component 204. at step 304. and determines the command type, at step 306. If the command 
type is "play", workstation 107 or processor 111 proceeds to step 308 to execute the command in the manner shown 
in FIG. 5: if the command type is "skip", workstation 107 or processor 111 proceeds to step 310 to execute thecommand 
in the manner shown in FIG. 4; and if the command type is "wait", workstation 107 or processor 111 proceeds to step 
312 to execute the command in the manner shown in FIG. 6. Following steps 308. 310, or 312, workstation 107 or 
processor 111 checks if the presentation script contains more commands, at step 312. If not. execution of the script 
ends, at step 316: if so, workstation 107 or processor 111 gets the next command of the presentation script, at step 
314, and returns to steps 306 et seq. 

The execution of a "skip" command is shown in FIG. 4. Upon encountering the "skip" command, at step 400, 
workstation 107 or processor 111 sets a skip indication, such as a memory flag or a register flag, at step 402. saves 
the argument of the "skip" command, at step 404. and then proceeds to process the next script command, at step 406. 

The execution of a "play" command is shown in FIG. 5. Upon encountering the "play" command, at step 500. 
workstation 107 or processor 111 retneves the component that is indicated by the argument of the play command, from 
message 200 in recipient's mailbox 105. at step 502. Workstation 107 or processor 111 then checks whether the skip 
indication (see step 402 of FIG. 4) is set. at step 504. If the skip indication is set. workstation 107 or processor 111 
gets the saved argument of the preceding skip command (see step 404 of FIG. 4). at step 506. and skips the amount 
of time or the number of records indicated by that argument into the message component that it retrieved at step 502. 
at step 508. This step may involve merely setting a "playback start" pointer to point the indicated amount of time or 
number of records into the message-body component from the beginning of the message-body component. Workstation 
107 or processor 111 then resets clears - the skip indication, at step 510 

Following step 510, or if it has been determined at step 504 that the skip indication is not set, workstation 107 or 
processor 111 checks whether the argument of the "play" command specifies a number of records, at step 512. If so. 
workstation 1 07 or processor 1 1 1 retains for presentation only that indicated number of the message-body component's 
records that follow the present position (i.e., the beginning of message, or see step 508) in the message-body com- 
ponent, at step 514. This step may involve merely setting a "playback stop" pointer to point the indicated number of 
records beyond the "playback start" pointer into the message-body component. If the argument of the "play" command 
does not specify a number of records, workstation 107 or processor 111 retains for presentation all records that follow 
the present position in the message, at step 516. This step may involve merely setting the "playback stop" pointer to 
point at the end of the message component Following step 514 or 516. workstation 107 or processor 11 1 proceeds to 
process the next script command, at step 518. 

The execution of a "wait" command is shown in FIG. 6. Upon encountering the "wait" command, at step 600, 
workstation 107 or processor 111 checks whether the command's argument specifies a time period, at step 602. If so. 
workstation 1 07 or processor 1 1 1 sets a timer to the specified time period, at step 604. If the argument does not specify 
a time period, but rather specifies a message-body component or a number of records of a message-body component, 
workstation 1 07 or processor 1 1 1 determines the playback duration of the specified component or records, at step 606. 
Illustratively, the duration is included among descriptive information that accompanies each component or record: 
alternatively, the duration is determined from the size of the file that constitutes the component or record. Workstation 
107 or processor 111 then sets the timer to the determined duration, at step 608. 

Following step 604 or 608, workstation 1 07 or processor 1 1 1 sends all previously-retained components (see steps 
514 and 516 of FIG. 5) to their playback mechanisms for simultaneous playback, at step 610. Illustratively processor 
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111 sends the retained components to workstation 107. while workstation 107 distributes the retained components to 
its display buffer, to a fax buffer, to a printer buffer, to a sound-card buffer, etc.. depending upon the type (i.e., medium) 
of each retained component. Workstation 1 07 or processor 1 1 1 then causes playback of the components to commence 
and starts the timer that it had set at step 604 or 608. at step 6 1 2. Workstation 1 07 or processor 1 1 1 monitors the timer, 
at step 614. When the timer expires, workstation 107 or processor 111 deletes the retained components that it had 
played at step 61 2. at step 6 16. This step may involve merely erasing the "playback start" and "playback stop" pointers 
into the message component, Following step 616. workstation 1 07 or processor 1 1 1 proceeds to process the next script 
command, at step6l8. 

To get a different presentation of the message, the message recipient merely selects and executes one of the 
other "presentation" components 204-205. 

Of course, various changes and modifications to the illustrative embodiment described above will be apparent to 
those skilled in the art. For example, different presentation component languages may be envisioned for defining the 
presentation scripts. Or. a different naming convention may be used to reference components. Likewise, conventions 
other than "records" may be used to define presentation elements, e.g.. "bytes", "sentences", and/or "pages". Further- 
more, messages may be allowed to have multiple components of the same media type. Such changes and modifications 
can be made without departing from the spirit and the scope of the invention and without diminishing its attendant 
advantages. It is therefore intended that such changes and modifications be covered by the following claims. 

Claims 

1. A messaging (102) apparatus having means (105) for storing a message (200) comprising a header component 
(201 ) that describes the message, and a plurality of body components (202-203) each carrying a portion of message 
information carried by the message. CHARACTERISED IN THAT 

the message further comprises a presentation component (204) separate from the header component and 
the body components that defines an order and any concurrency of presentation of the body components to 
a message recipient: and 
the messaging apparatus further comprises 

means (111:300-318) responsive to invocation of the presentation component for presenting the body com- 
ponents to the message recipient in the order and with any said concurrency defined by the presentation 
component. 

2. The messaging apparatus of claim 1 wherein: 

the message comprises a plurality of presentation components (204-205) each defining a different order or 
concurrency of presentation of the body components: and 

the presenting means are responsive to invocation of an individual one of the presentation components, for 
presenting the body components to the message recipient in the order and with any said concurrency defined 
by the individual one presentation component. 

3. The messaging apparatus of claim 1 wherein: 

the message comprises a plurality of body components (202-203) at least some of which are expressed in 
different media, and the presentation component defines an order of presentation of the body components 
including a concurrency of presentation of the body components expressed in the different media: and 
the presenting means are responsive to the invocation of the presentation component, for presenting the body 
components in the order defined by the presentation component including presenting concurrently the body 
components expressed in the different media. 

4. The messaging apparatus of claim 1 wherein: 

the presentation component further defines durations of presentation of the body components: and 
the presenting means are responsive to the invocation of the presentation component, for presenting the body 
components in the order and with any said concurrency for the durations defined by the presentation compo- 
nent. 



5. The messaging apparatus of claim 1 wherein: 
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at least one of the body components (202) comprises a plurality of records (206). and the presentation com- 
ponent defines an order of presentation of the records: and 

the presenting means are responsive to the invocation of the presentation component, for presenting the 
records in the order defined by the presentation component. 

The messaging apparatus of claim 1 wherein: 

a plurality of the body components (202-203) are each expressed in a different medium and each comprises 
a plurality of records (206). and the presentation component defines an order of presentation of the records 
including any concurrency of presentation of records expressed in different media: and 
the presenting means are responsive to the invocation of the presentation component, for presenting the 
records in the order and with any said concurrency defined by the presentation component. 

The messaging apparatus of claim 6 wherein: 

the presentation component further defines durations of presentation of the records: and 

the presenting means are responsive to the invocation of the presentation component, for presenting the 

records for the durations defined by the presentation component. 

A messaging method comprising the step of receiving in messaging equipment (102) of a message recipient a 
message (200) that comprises a header component (201) describing the message, and a plurality of body com- 
ponents (202-203) each carrying a portion of message information carried by the message. CHARACTERISED 
IN THAT 

the message (200) further comprises a presentation component (204) separate from the header component 
and the body components that defines an order and any concurrency of presentation of the body components 
to the message recipient: and 
the method further comprises the step of 

in response to the message recipient invoking the presentation component in the messaging equipment, the 
messaging equipment presenting (300-318) the body components to the message recipient in the order and 
with any said concurrency defined by the presentation component. 



The messaging method of claim 8 further comprising the steps of: 



creating (101. 106) the message, including creating the presentation segment, under direction of a human 
message originator: and 

sending (100) the message from the message originator to the message recipient. 
The messaging method of claim 8 wherein: 
the step of receiving comprises the step of 

receiving in messaging equipment (102) of a message recipient a message (200) that comprises a header 
component (201) describing the message, a plurality of body components (202-203) each carrying a portion 
of message information carried by the message and at least some of which are each expressed in a different 
medium, and at least one presentation component (204-205) separate from the header component and the 
body components, each presentation component defining an order and durations of presentation of the body 
components to the message recipient, including defining concurrency of presentation of at least some of the 
body components expressed in different media: and 
the step of presenting compnses the step of 

in response to the message recipient invoking (300) an individual presentation component in the messaging 
equipment, the messaging equipment presenting (302-318) the body components to the message recipient 
in the order and for the durations defined by the individual presentation component. Including presenting said 
some of the body components concurrently to the message recipient. 
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FIG. 6 
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(54) Apparatus and method for specifying presentation of multimedia message components 



(57) A new type of message component (204. 205) 
enables message creators to specify, inside of a mes- 
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